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^H ' Abstract 



We show that the center of mass of Ising vectors that obey some simple 
constraints, is again an Ising vector. 



*vj ' Many problems in statistical mechanics are formulated in terms of an N- 

^ . dimensional vector J, with components J,;,i — 1,...,N that take only binary 

Cs| ' values Ji = ±1. We will call such a vector an Ising vector. Its components 

^^ . represent for example a spin state (Ising model [Q), the occupancy of a site 

I~J ' etc. In the thermodynamic limit N — > oo, only a subset of all the possible 

^— V , configurations {J} are typically realized. In many cases, they are characterized 

O^ ' by simple contraints of the form : 

"ti . lim = R lim = q, (1) 

B 



where J and J' are typical members of the subset, B is a symmetry breaking 
direction (imposed from the outside or arising through a phase transition), while 
(— I ' q and R are physical properties describing the resulting macroscopic state (for 

Q , example the magnetization or the density). In this letter, we focus on the 

O ■ center of mass of the vectors J, that satisfy the above constraints. We report 

the surprising finding that it is an Ising vector whenever B is Ising. 

To construct the center of mass we follow a Monte Carlo approach by choos- 
ing at random n vectors J", a — 1, ...,n, that satisfy the constraints, and con- 



C^ ' sidering their center of mass: 

Y = C-l^J^ (2) 

a 

with the proportionality factor C = ^n + n(?i — \)q^ so that the normalization 
condition Y^ = A^ is obeyed. In general the vector Y is not Ising, but our 



contention is that it becomes Ising in the hmit n — + oo, provided B is Ising. 
In view of the permutation symmetry between the coordinate axes, it will be 
sufficient to prove that BiYi ~ C~^ J^a ^i "^i '^^^V takes the values +1 and 
— 1 in this limit. To show that this is the case, we focus our attention on the 
probability density P{y) of the variable y = n~^ J^a -^i "^f ' which differs by a 
factor -nT^ ^n + n(n — \)q "l4°° ^ from B{Y\. It is given by: 



nv) 



J|dJ"P6(J'') 5(J'' • B - NR) 
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(3) 



where Pi, is the measure restricting to vectors with binary components. 



n(J) 



N 

n 



-6{Jj - 1) + -d{Jj + 1) 



(4) 



and the proportionality constant has to be determined from the normalization 
condition J dy P{y) = 1. The r.h.s. of (0) resembles an ordinary replica 
calculation [g[ , but with as limit of interest the number of replicas n tending to 
infinity. 

Rather than following the standard but lengthy calculations that are usual 
in this case, we present a more elegant, direct and expedient procedure. Since 
y — n^^'^a^a, with Xa = JiBi, we evaluate the joint probability density 
P(x) of the n-dimensional vector with binary components Xa, a — l,...,n. 
Since all choices of the vectors J" that satisfy the constraints can be realized, 
the Shannon entropy is maximized under the constraints (]^. Hence P(x) is 
found by maximizing Q] its Shannon entropy — ^^ Pi^) lii^(^)j subject to the 
constraints: 
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{a < b). 



One finds: 



P(x) = Z-^ exp 



y^ RaXa + ^ QabXa 



Xb 



a<b 



(5) 



(6) 



where Z follows from the normalization of -P(x). The values of the Lagrange 
multipliers {Ra} and {qab} have to determined from the constraints (ra). In 
view of the permutation symmetry in the replica indices, -Ra and qab must be 
independent of a and 6, Ra — R, qab — Q, rendering the evaluation of Z very 
simple: 

Z{R, q) = e-"«/2 f Dz [cosh (r + zy/q\T , (7) 



while (||), determining R and q, reduce to: 



R 



2 Q J du exp — (u — _R)^/2g (coshw)" tanhu 

\nZ = 

n OR 



J du cxp —{u — RY/2q (coshw)" 



n{n — 1) dq 



a J du exp —{u — R)'^/2q (cosh u)" tanh^ u 
— InZ = ^ , ^ , .(8) 



J du exp —{u — RY/2q (coshw)" 



As a resuh of the "rephca symmetry", we conclude from (g) that P(x) is in fact 
a function of ^^ Xa = ny. Hence P{y) is obtained from P(x) by multiplication 
with a combinatorial factor, expressing the freedom to choose which n(l + y)/2 
components of x are +1 and which remaining ones are —1, for a given total 
value ny of their sum: 
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n{l+y) ) exp 
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n qy 



(9) 



y can take the values — 1, — 1 + 2/n, . . . , 1 — 2/n, 1, and the proportionality con- 
stant is again fixed by normalization. This result is in agreement with a direct 
evaluation of (ra) but very different from the result for continuous components 
discussed in [|j. Unfortunately, the above expression is quite complicated, espe- 
cially in view of the fact that we did not succeed in solving explicitly the eqs. (|h 
determining the Lagrange multipliers. Concordantly, the components of Y are 
not binary for any finite n. As an illustration, we have included in fig. yj the 
results obtained by a numerical solution of (g) for the special case q = R and 
several values of n. The corresponding results for the probability density for y 
(or equivalently, BiYi), are plotted in fig. Q 

In order to extract the asymptotic behavior for the n —^ oo limit, one needs 
to guess the asymptotic dependence on n of the Lagrange parameters. The 
correct scaling appears quite naturally in the calculations for the simpler case 
of vectors J with continuous components. Here, we just note by inspection that 
the eqs. (g) for the properly scaled Lagrange parameters p = nR and "f = nq 
read: 



R 



where 



j due "'^(") smh{up/^) tanh u 

J du e-"'^(") cosh(up/7) 

/ du e-"'^(") cosh(up/7) tanh^ u 

J du e-""^(") cosh{up/j) 



u 

2^ 



In cosh 1 



(10) 



(11) 



The appearance of the hyperbolic functions of up/j in eqs. (nfl) is due to the 
fact that (f) is even. The saddle point approximation can now, for n — > oo. 
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Figure 1: In order to account for a fogarithmic divergence in the limit q 
exp(— 7), 7 = nq, is plotted as a function of q for several values of n. 



1, 



be applied in a straightforward manner on the u-integrations, leading to the 
following simple and explicit solutions for the scaled Lagrange variables: 



arctanhy^ 
arctanh(_R/Y^) 



(12) 



Inserting this result together with the asymptotic expression for the combina- 
torial factor in (H), one finally obtains the following asymptotic result for P{y): 
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l(l + ^),fe-^)+l(l-^,,to + ^), ,13) 



In view of the aforementioned relation between y and the (first) component of 
Y, the convergence of the latter to an Ising vector follows immediately. 

As a first application of the above result, we turn to the case of an Ising 
spin system in the ferromagnetic phase. Choosing the vector B with all its 
components equal to 1, we note that R plays the role of the magnetization. All 



the spin states J are otherwise allowed, and lie on the rim of the N dimensional 
sphere with radius \N at fixed angle arccosi? with B. It is thus clear that the 
center of mass is B itself. This trivial result is recovered from (nsh by noting 
that q = R^ va this case. Note that the constraint J" • J'' = Nq is therefore 
redundant, which implies ^ = and R = arctanhi? (a result valid for any n). 

A case of special symmetry \s q ^ R. In this scenario, no macroscopic 
measure allows to distinguish between the symmetry breaking direction B and 
each of the vectors J°. The Lagrange parameters also present the symmetry 
R = q, which can be seen from eqs. (H). 

A third case of interest is the limit R ^ while q remains finite. In this 
case, the J-vectors lie in the subspace orthogonal to B and satisfy as single 
constraint the prescribed mutual overlap q. From eqs. (j^ it is clear that R = 
is automatically satisfied, and one concludes from (|3|) that the center of mass 
of the J-vectors is again an Ising vector (but its components are equally likely 
to be +1 or —1). 

It is very tempting to apply the results above to neural network learning 
problems B where a student perceptron J learns from examples generated by a 
teacher perceptron B ^, 0, g] • Indeed, so-called Gibbs learning |ll| presents the 
symmetry q = R, and the interest of the center of mass is that it is, according 
to a simple general argument |Q], see also M, the "best" student having the 
largest overlap with the teacher (namely \AR). Accordingly, R = and q y^ 
are constraints satisfied by Ising vectors which solve the capacity problem IJI4 . 
However, these are disordered systems, and the conditions on R and q alone 
do not convey all the information which is necessary to describe the constraints 
in J space. Therefore, even though the constraints (El) are satisfied in neural 
network problems, result ( p3| ) does not apply to them. 

We conclude with a verification of the theoretical prediction (1^) by run- 
ning simulations for the mean field ferromagnetic Ising spin model, fig. pi The 
Metropolis algorithm was allowed to run for a number of Monte Carlo steps per 
site (MCS) until thermalization was considered to be achieved. Then vectors 
were sampled every 5 MCS (to allow sufficient decorrelation between consecutive 
samplers) and summed to construct the center of mass. The small discrepancy 
for N = 100 with the theoretical prediction is due to finite size effects. For 
N = 1000, the results are nearly indistinguishable on the scale of the figure 
from the theoretical values. It is interesting to note that the hard constraints 
of eq. (|l]) are satisfied only in the thermodynamic limit. In the simulations R 
(and q) are distributed with peaks whose width scales with N~^''^. Nonetheless 
the effect of these fiuctuations on the resulting P{BiYi) is negligible. 
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Figure 2: Probability density for BiYi according to eq. 
several values of n. The legend is the same for both plots. 
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Figure 3: Probability density for BiYi in the q = R^ scenario and several values 
of n. The lines represent the theoretical curve (0), while points represent results 
from a simulation of the mean field ferromagnetic Ising model with N = 100, 
dimensionless temperature 1.09 and dimensionless magnetic field 0.1 (amounting 
to a magnetization R ~ 0.5 - see text for details). Inset: second cumulant (cr^) 
of the distribution as a function of 1/n. The squares represent the simulations, 
while the dashed straight line corresponds to the theory (in the asymptotic limit 
n^oo, eq. (O) implies cr^ ^ (1 - R^)/{nR^)). 



